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Abstract. We prove the Strassen's strong invariance principle for vector-valued 
additive functionals of a Markov chain via the martingale argument and the theory 
of fractional coboundaries. The hypothesis is a moment bound on the resolvent. 



1. Introduction 

Let (X n ) n > denote a stationary ergodic Markov chain denned on a probability 
space (fl,^, P), with values in a measurable space (X,B). Let Q(x,dy) be its 
transition kernel and ir the stationary initial distribution. Furthermore, for p > 1, 
let L p (ir) denote the space of (equivalence classes of) £>-measurable functions g : 
X — > M. d for some d > 1 and such that \\g\\p := f x \g(x)\ p n(dx) < oo, and let Lq(tt) 
denote the set of g G L p (ti) for which f x gdn = 0. Here, | • | denotes the Euclidean 
norm on M. d . 

Now, fix d and an K. d - valued function g G Lq(tt). For n > 0, define 

n 

S n+1 = S n+1 (g) := ^2,g{Xi) and 5*0 = 0. (1.1) 

i=0 

For the question of central limit type results for S n , there have been numerous studies 



from many angles and under different assumptions; see Maxwell and Woodroofe [10| 
Derriennic and Lin j^] [6| and references therein. In this note, we mainly consider the 
iterated logarithm type results for S n . Since the appearance of Strassen's paper 



almost sure invariance principles for the law of iterated logarithm have been obtained 



for a large class of independent and dependent sequence (Y n ) n >i; see Strassen [19 
Hall and Heyde and Philipp and Stout [l6j]. Here, the Skorokhod representation 
plays an important role. 

It is well known that, the law of the iterated logarithm (in short LIL) is closely 
related to the central limit theorem (in short CLT) in some sense. There are several 
approaches to these problems. If the chain is Harris recurrent, then the problems 
may be reduced to the independent case in a certain sense, see Meyn and Tweedie 
[llf and Chen [3|. If there exists a solution to Poisson's equation, h = g + Qh, then 
the LIL and CLT problems may be reduced to the martingale case, see also Gordin 
and Lifsic 0] and Meyn and Tweedie llj]. Bhattacharya 0] obtained the functional 
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CLT and LIL for er go d ie stationary Markov processes by discussing the infinitesi- 
mal generator. Wu [20(] extended the forward-backward martingale decomposition 
of Meyer-Zheng-Lyons's type from the symmetric case to the general stationary 
situation and gave the Strassen's strong invariance principle. 

Our goal, in this paper, is to consider the problem that S n satisfies the LIL under 
some proper conditions. Note that, Rassoul-Agha and Seppalainen [l7| mainly relied 
on the invariance principle for vector-valued martingales, so it is likely to obtain the 
invariance principle for LIL for the vector-valued additive functionals of a Markov 
chain, only if we can develop the corresponding theory for vector- valued martingales. 
However, we encounter the essential difficulties, when considering the vector-valued 
martingale, since Monrad and Philipp [l3[ proved that it is impossible to embed 
a general IR d -valued martingale in an M d -valued Gassian process. For the strong 
approximation of random sequence taking values in general Banach space, please 
refer to Philipp [l5| and the references given there. 

In the present paper, we will take along the lines of Kipnis and Varadhan [9[ and 



Maxwell and Woodroofe [10|, to the case where a solution is not required. Moreover, 
we identify the lim sup in LIL just the square root of the trace of the diffusion matrix 
corresponding to the functional CLT. 

Let us explain the organization of this paper. In Section 2, we state our main 
results. Section 3 gives the proof of our main results mentioned in Section 2, 
which mainly depends on the strong approximation of vector-valued martingales 
(see Berger and the theory of fractional coboundaries developed by Derriennic 
and Lin P. 



2. Main results 

For a function h G L 1 ^), and 7r-a.e. x G X define an operator 

Qh(x) = j h(y)Q(x,dy). (2.1) 

Obviously, Q is a contraction on L p (7r) for p > 1. For e > 0, let h £ be the solution 
of the equation 

(l + e)h = Qh + g. 

In fact, 

oo 

h £ = Y,^ + e)- n Q n ~ l g. (2.2) 

71=1 

Note that h £ G L p (tt), if g G L p (ir). Let 7Ti be the joint distribution of Xq and X±, 
so that 7ii(dxo,dxi) = Q(xo, dxi)ir(dxo); denote the L 2 -norm on L 2 {jii) by || ■ ||i; 
and let 

H 6 (x ,xi) = h £ (xi) - Qh £ (x ) 
for x , X\ G X . For any e > 0, let 

n-l 

M n {e) = Y,H £ {X l ,X i+l ) and R n (e) = Qh £ (X ) - Qh £ (X n ), 

i=0 
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hence, by the simple computation, 

S n (g) = M n (e) + eS n {h £ ) + ^(e). (2.3) 

For convenience, we summarize the results of Maxwell and Woodroofe 10| as the 
following theorem. 

Theorem MW Assume that g G Lq(tt) and that there exists an a G (0, 1/2) such 
that 

n-l 

\\J2Q l 9h = 0(n a ). (2.4) 

i=0 

Then we have 

(1) The limit H = lim e ^ + H e exists in L 2 (ni). Moreover, if one defines 

n-l 
i=0 

where mi = H(Xi, X i+ i) , then (m n ) n > is a stationary and ergodic F-square 
integrable martingale difference sequence, with respect to the filtration \T n = 
°~{Xo-, • • • , X n )} n > ; 

(2) \\h £ \\ 2 = 0(e~ a ), and if R n = S n - M n = M n (e) - M n + eS n (h £ ) + R n (e), 
then 

E(\R n \ 2 ) = 0(n 2a ). 

Remarks 2.1. Furthermore, if there exists a p > 2 such that g e L p (tt), then there 
exists age (2,£>) such that H e L q (tt) and (M„) n >\ is an L q -martingale; see the 
Theorem 1 of Rassoul-Agha and Seppalainen \l 7] . 

For introducing our main results, we need give a few more notations. Let C([0, 1], M d ) 
be the Banach space of continuous maps from [0, 1] to M. d , endowed with the supre- 
mum norm ||| • |||. using the Euclidean norm in Mr. Denote K the set of absolutely 
continuous maps / G C([0, 1], M d ), such that 

/(0) = 0, f \f(t)\ 2 dt<l, 



where, / denotes the derivative of / determined almost everywhere with respect to 
Lebesgue measure. Obviously, K is relatively compact and closed. 
Define 

U(t) = (2nloglogn)- 1/2 [5 fe + (nt - k)g(X k )} 

for t G [^, ^-), k — 0, 1, 2, • • • — In order to avoid difficulties in specification, 
we adopt the convention that log log a; = 1, if < x < e e . Then, £ n is a random 
element with values in C([0, 1], M. d ). 

After these preparations, we are now in a position to state our main results. 

Theorem 2.2. Let g G L^{-k) (p > 2) and assume that there exists an a G (0, 1/2) 
for which \2.J$ is satisfied. Then, the sequence of functions (£ n ( - ); n > 1) is rela- 
tively compact in the space C([0, 1], M. d ), and the set of its limit points as n — > oo ; 
coincides with ^tr(Q)K, where tr(-) denotes the trace operator of a matrix and 



4 GUANGYU YANG AND YU MIAO 

D = E(M 1 M-[) = J HH l dTTi is the diffusion matrix corresponding to the functional 
central limit theorem; see Rassoul-Agha and Seppalainen \l i a 1. 

Theorem 2.3. Let g G Lq{^) (p > 2) and assume that there exists an a G (0, 1/2) 
for which {2.1$ is satisfied. Then 

limsup | S n \/ a/272 log log n = \/tr(2)) P-a.s. (2.5) 

Remarks 2.4. In fact, tr(S) = ^4nd particularly, if putting d = 1, we can 

obtain the main results of Miao and Yang flBj . 

Remarks 2.5. Forn > 0, define S* = S n — Kx S n , since the Theorem 3 of Rassoul- 



Agha and Seppalainen flnj . the above Theorem 2.1 and Theorem 2.2 also hold for 



n ' 



3. Proof of main results 
3.1. Proof of Theorem [2H1 

Proof. For < t < 1, define 

CnO) = (2nloglogn)" 1/2 M [n f], 
rj n (t) = (2nhg\ogny 1/2 B(nt), 

where, M n is as defined in Section 2 and B(-) is an R -valued Brownian motion with 
mean and diffusion matrix D. Theorem 1 of Strassen 1^| shows that (r] n (-)) n >i is 



relatively compact and the set of its limit points coincides with A/ / tr(S))A'. 

Notice that by the part (1) of Theorem MW, (M n ) n >i is a square integrable 
martingale with strictly stationary increments. Moreover, 

E(< u, m > 2 ) < oo and E(< u, m >) = 0, for all u G R d , (3.1) 

where, < -, • > denotes the inner product in M. d . Therefore, Corollary 4.1 of Berger 
[l| implies that, 

Without changing its distribution, one can redefined the sequence (M n ) n >i on a 
new probability space (Cl, j 7 , P) on which there exists an M. d -valued Brownian motion 
(B(t)) t >o with mean and diffusion matrix D such that 

\M [t] - B(t)\ = o((tloglogt)- 1/2 ), P - a.s. (as t -> oo). (3.2) 

where, 2) = lim^oo n Cov(M n ). 

Remarks 3.1. Birkhoff-Khinchin's ergodic theory and together with the simple cal- 
culation shows that, D = E(M]M*) = J H^dni, is the diffusion matrix correspond- 
ing to the functional central limit theorem; see also Rassoul-Agha and Seppalainen 



12]. 

That is to say, 



sup \M [nt ] - B(nt)\ = o((2nloglogn)" 1/2 ), P-a.s. 

0<i<l 
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Hence, 

IHCn-^IH =(2nloglogn)- 1/2 sup \M [nt] - B(nt)\ 

0<t<l 

=o(l), F-a.s. 

Define 

C n (t) = (2nloglogn)- 1 / 2 [Af fe + (nt - k)m k ) 
for t e g, *±1), k = 0, 1, 2, • • • , n - 1. Then C„ e C([0, 1], M rf ) and 
sup |C„(t) - Cn(*)| = (2nloglogn)~ 1/2 max \m k \. 

tG[0,l) 0<fc<n-l 

Next, we give the order estimation of maxo<fc< n _i \m k \. 

Lemma M ( See Moricz 11$ ) Let p > and (3 > 1 be two positive real numbers and 
Zi be a sequence of random variables. Assume that there are nonnegative constants 
dj satisfying 

i i 
3=1 3=1 

for 1 < i < n. Then 

i n 

E(max \J2 Z 3\ P ) < C P , g (E a ^> ( 3 - 4 ) 
j=i i=i 

for some positive constant C v $ depending only on p and (5. 

Lemma 3.2. For any enough large n, there exists a positive constant C such that 

E(max \mi\ q ) < CE\m 1 \ g . (3.5) 

l<i<n 

Proof. Since the part (1) of Theorem MW and Remarks 2.1, let E|mi| g = a 2 (q) and 
for any k > 1, we have the following relations, 

k 

E|m fc | 9 < (^a*) 2 , (3.6) 

i=l 

where a\ = a(q) and a, = for 2 < % < k. Hence, by Lemma M, there exists a 
constant C > 0, such that 

n 

E(max \mi\ q ) < C(V a;) 2 = CE(\ mi \ 9 ). (3.7) 

Ki<n 

i=l 

This completes the proof of the lemma. □ 
For any e > 0, Lemma 3.2 immediately yields, 

P( max \m k \ > e(2nloglogn) 1 ^ 2 ) = 0((nloglogn)~^ 2 ). 

0<fe<n— 1 

By Borel-Cantelli's lemma, we have 

(2nloglogn) _1//2 max \m k \ — o(l), P — a.s. (3-8) 

0<fc<n— 1 
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Hence, 

|||Cn-C«l||= SUp |Cn(t) -Cn(*)| =o(l), P - d.S. 

te[o,i) 

The above discussions immediately yield the following claim: 

(Cn('), n > 1) is relatively compact and the set of its limit points coincides with 

Now, we turn to deal with the neglectable term R n in the sense of functional LIL. 
Firstly, let us recall the concept of Dunford-Schwartz (DS) operator; see Derriennic 
and Lin Q]. We call T a DS operator on L 1 of a probability space, if T is a contraction 
of L 1 such that ||r/||oo < 1 1/ 1 loo for every / G L°°. If 6 is a measure preserving 
transformation in a probability space (Q, then the operator Tf = f o 9 is a 

DS operator on More generally, any Markov transition operator P with an 

invariant probability measure yields a positive DS operator. 

Lemma DL ( See Derriennic and Lin fyj) 

(1) Let T be a contraction in a Banach space X, and let < (3 < 1. If 

1 n 

sup |1 — — y^T fc y|| < oo, i/ien y E (I - T) a X for every < a < (3. 
n n ' k=i 

(2) Lei T is a DS operator in L 1 (/i) o/ a probability space, and fix 1 < p < oo, 
with dualq = p/(p-l). LetO<a<l, and f G (I-T) a L p . If a > l-± = \, 

- n— 1 

t/jen — y- VT fc / ->0 a.e. 

k=0 

To apply the above lemma, i.e., the theory of fractional coboundaries named by 
Derriennic and Lin jij, we need construct a DS operator. On X x X, define 

/Oo, a*) = flf(sc ) - #(^0, 

then 

n-l 

= J]^(X i )-iJ(X i ,X m )] 

i=0 
n-l 

= ^/(X,,X m ). (3.9) 

i=0 

Let # be the shift map on the path space X n for the Markov chain which is a 
contraction on L 2 (P). Hence, 6 is a DS operator. For a sequence cc = (xj)j g N G /f N , 
define F(a;) = f(x ,Xi), then we have 

n-l 



FGL 2 (P) and R n = ^Fo6 k . 



k=0 
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From the part (2) of Theorem MW, there exists a constant 1/2 < (3 < 1 — a, such 
that 



n-l 

I 

sup 



n 



_^Fo^ <oo, (3.10) 



rr 

fc=o 



Since the part (1) of Lemma DL and < a < 1/2, we have F E (I — 6 I )' 7 L 2 (P), for 
some ?7 G (1/2, 1 — a). By the part (2) of Lemma DL, we have 

-rj.Rn^O, F-a.s. 

Furthermore, applying an elementary property of real convergent sequences, we 
immediately get 

max \R k \ = o((2n loglogn) 1 / 2 ), P — a.s. 

0<k<n 

Consequently, 

(2nloglogn)- 1/2 sup \R [nt] \ -»• 0, P - a.s. (3.11) 

0<t<l 

From above discussions, we complete the proof of Theorem 12.21 □ 
3.2. Proof of Theorem [HH 

Proof. Here, we take along the lines of the proof of Theorem 4.8 in Hall and Heyde 
Let {e,}f =1 the canonical basis of R d . For any valued function /, denote 
/ = (/i, /2, • • • , fdY- By the definition of K, we have, for any / G ^/tr(D)K, 

i/wi a = E(/*^ a ) da ) a 
i=i - 70 

d /-t /-i 

<V(/ /;(s) 2 ds) / lds<tr(D)t (3.12) 
where, the first inequality by the Cauchy-Schwartz's inequality. So, < \Jtr{T))t. 



It follows that sup t6 r 01 i < y/tr(D). Hence, by Theorem 2.1, 

limsup sup |f n (t)| < y/tr(J3), F-a.s. (3.13) 

te[o,i] 

and setting i = 1, 

limsup 1 5„ |/V2n log log n < A/tr(S)), P - a.s. (3.14) 



On the other hand, we put f(t) = tJ ^) J2t=i e h t G [0, 1]. Then, / G y/tr{®)K 
and so for P — a.s. u, there exists a sequence n k = n k (uj), such that 



Particularly, /(l) = J^^tx^i, |£ Bfc (l)( w )| — |/(1)|. That is to say, 



I^J/v^nUoglognfc = y/ti(®), P - a.s. (3.16) 
This completes the proof of Theorem 12.31 □ 



8 



GUANGYU YANG AND YU MIAO 



Acknowledgements 

The authors wish to thank Prof. Y. Derriennic and Prof. M. Lin for sending the key 
reference paper 0]. 



References 

[I] Berger, E. (1990). An almost sure invariance principle for stationary ergodic sequences 
of Banach space valued random variables. Probab. Th. Rel. Fields 84, 161-201. 

[2] Bhattacharya, R. N. (1982). On the functional central limit theorem and the law of 
the iterated logarithm for Markov processes. Z. Wahrs. verw. Geb. 60, 185-201. 

[3] Chen, X. (1999). Limit theorems for junctionals of ergodic Markov chains with general 
state space. Memor. Amer. Math. Soci. No. 664. 

[4] Derriennic, Y. and Lin, M. (2001). Fractional Poisson equations and ergodic theorems 
for fractional coboundaries. Israel J. Math. 123, 93-130. 

[5] Derriennic, Y. and Lin, M. (2001). The central limit theorem for Markov chains with 
normal transition operators, started at a point. Probab. Th. Rel. Fields 119, 508-528. 

[6] Derriennic, Y. and Lin, M. (2003). The central limit theorem for Markov chains started 
at a point. Probab. Th. Rel. Fields 125, 73-76. 

[7] Gordin, M. I. and Lifsic, B. A. (1978). The central limit theorem for stationary Markov 
processes. Dokl. Akad. Nauk SSSR 19, 392-394. 

[8] Hall, P. and Heyde, C. C. (1980). Martingale limit theory and its application. Acad. 
Press, New York. 

[9] Kipnis, C. and Varadhan, S. R. S. (1986). Central limit theorem for additive functionals 
of reversible Markov processes and applications to simple exclusions. Comm. Math. Phys. 
104, 1-19. 

[10] Maxwell, M. and Woodroofe, M. (2000). Central limit theorems for additive function- 
als of Markov chains. Ann. Probab. 28 (2), 713-724. 

[II] Meyn, S. P. and Tweedie, R. L. (1993). Markov chains and stochastic stability. 
Springer- Verlag, New York. 

[12] Miao, Y. and Yang, G. (2006). The law of the iterated logarithm for additive func- 
tionals of Markov chains. Submitted. 

[13] Monrad, D. and Philipp, W. (1990). The problem of embedding vector-valued mar- 
tingales in Gaussian process. Teor. Veroyatn. Primen. 35, 384-387. 

[14] Moricz, F. (1976). Moment inequalities and the strong laws of large numbers. Z. 
Wahrs. verw. Geb. 35, 299-314. 

[15] Philipp, W. (1986). Invariance principles for independent and weakly dependent ran- 
dom variables. In: Dependence in Probability and Statistics. Proc. Conf. Oberwolfach 
1985. Boston Basel Stuttgart: Birkhauser. 

[16] Philipp, W. and Stout, W. F. (1975). Almost sure invariance principles for partial 

sums of weakly dependent random variables. Memor. Amer. Math. Soci. No. 161. 
[17] Rassoul-Agha, F. and Seppalainen, T. (2004). An almost surely invariance principle 

for additive functionals of Markov chains. arXiv: math. PR/0411603. 
[18] Strassen, V. (1964). An invariance principle for the law of the iterated logarithm. Z. 

Wahrs. verw. Geb. 3, 211-226. 
[19] Strassen, V. (1965). Almost sure behavior of sums of independent random variables 

and martingales. Pro. 5th Berkeley Symp. Math. Stat, and Probab. 2, 315-344. 
[20] Wu, L. M. (1999). Forward-backward martingale decomposition and compactness 

results for additive functionais of stationary ergodic Markov processes. Annals de I'i. H. 

P., Serie Probab. and Stat. 35 (2), 121-141. 



INVARIANCE PRINCIPLE FOR LIL FOR MARKOV CHAINS 



9 



School of Mathematics and Statistics, Wuhan University, 430072 Hubei, China 
E-mail address: study _yang@yahoo . com . cn 

School of Mathematics and Statistics, Wuhan University, 430072 Hubei, China 
and School of Mathematics and Information Science, Henan Normal University, 
453007 Henan, China. 

E-mail address: yumiao728@yahoo.com.cn 



